Infrared Imager Readout Electronics

ABSTRACT

Readout integrated circuits placed below the suspended sensor elements detect changes of electrical resistance of sensor elements and digitize the signals with digital to analog convertor for each element. Readout electronics provides low parasitics, high signal to noise ratio, high data rate, high dynamic range and instantaneous global readout.

RELATED U.S. APPLICATION DATA

Provisional application No. 61/704,145 filed on Sep. 21, 2012.

BACKGROUND

Micro-Electro-Mechanical Systems (MEMS) microbolometers arewavelength-independent detectors that sense incident electromagneticradiation by the temperature increase caused by the radiation'sabsorption in sensing elements. The sensing element includes atemperature-sensing material whose resistivity is dependent ontemperature. The temperature (or rather temperature change) of theelement then can be read-out by measuring the resistance of sensingelement using associated circuitry. Detectors can be used as singleelements to monitor temperature or arrayed in a focal plane array (FPA)to form an image.

Microbolometers are typically optimized to detect infrared wavelengthsin the 2-14 μm region where traditional photonic sensors are insensitive(as in the case of silicon-based charge-coupled device (CCD) orcomplementary metal-oxide semiconductor (CMOS) image sensors) orexpensive to fabricate (as in the case of quantum-well devices). Theycan be used in cameras that have applications in night vision,surveillance/security, medical imaging, and search and rescue.Alternatively, single elements or several elements can be used fornon-contact temperatures sensing in mobile phones and other devices.

Relative to visible light imaging, infrared (IR) imaging using MEMSmicrobolometers suffers serious shortcomings in image and videoperformance. In addition to the fact that current IR imaging greatlylags visible imaging in resolution, modern IR imagers exhibitinsufficient dynamic range and insufficient grey-scale allocated toareas of interest such as human subjects or other warm objects. Inscenes with motion, scrolling shutter artifacts occur such as wobble,skew, smear, partial exposure, and aliasing. Because of readout andsensor limitations, capture times are long and consequently, frame ratesare low. Finally, de-noising, image enhancement, and imagepost-processing are inadequate. While performance of the microbolometerstructure itself has made advancements with new designs, materials andfabrication methods, implementation of improved readout technology hasyet to follow to match sensor gains.

Furthermore, although infrared imaging using microbolometers has foundwidespread applications in military, industrial and consumer products,their use has generally been limited to high-cost, low-volume productsprimarily because of the high cost of the microbolometer imager itself,which can account for about 50% of the total imaging system cost. Majorcontributors to the imager cost are the relatively large pixel sizerequired to achieve acceptable device sensitivity and typically highyield loss due to pixel to pixel performance variation (among othercontributors). State-of-the-art microbolometer pixel pitch is currently17 um, or roughly 200 times the area of state-of-the-art visible-lightCMOS image sensor pixels. Such relatively large pixel size results in alarge array area, a large die size, and therefore fewer die-per-wafer,lower yields and high cost. Furthermore, a large array necessitateslarger optics and optical paths which contribute to larger and moreexpensive systems. Therefore, improved yield and smaller pixel size canreduce imager and system costs in various ways enabling adoption ofmicrobolometer infrared imaging into more price sensitive and highervolume products.

SUMMARY

Design, architecture, and implementation of readout integrated circuitrythat is capable of detecting and measuring very small changes inresistance (and consequently small changes of voltage or current) in adevice such as a sensor is disclosed. The readout circuitry performs theanalog measurement and subsequent analog to digital conversion (ADC) athigh-frequency while maintaining a footprint of in 100 square micronrange. The small readout circuitry footprint allows its implementationwithin each single pixel (in-pixel ADC) of a microbolometer focal planearray enabling improved and novel capabilities including:

1) Fast readout and multiple image captures2) Global shutter capability3) Expanded dynamic range where needed4) Pixel-by-pixel calibration and compensation5) Improved noise reduction process, especially 1/frequency (1/f).

Fast sampling and frame capture in conjunction with global shutter thatare uniquely possible with in-pixel ADC allow for high frame rates andreduced motion artifacts enabling motion and gesture sensingapplications such as automotive night vision and gaming.

Fast sampling also enables the unique capability of initial sub-samplingof each pixel and subsequent dynamic setting of the integration time ona pixel-by-pixel basis within the period of a single frame. Therefore,the integration time for each pixel can be customized during each frame,based on the pixel's brightness. This allows expansion of the dynamicrange around any region of interest. Memory registers used for storageof individual pixel integration times can also be linked to non-volatilememory registers that store the predetermined ADC schedules which mapthe DAC voltage sub-ranges to the different integration time. Theschedules are based on the average IR intensity level, contrast, type ofscene, and different applications. These values are also adjusted duringa power-on calibration routine using a shutter. In this manner, pixelresponse can be calibrated on a pixel-by-pixel basis, therebycompensating for variation arising from fabrication. Such compensationreduces one of the main sources of chip-sort yield loss.

Finally, fast sampling, frame capture, and storage are essential fornoise cancelation and reduction using auto-correlation methods in whicha pixel is probed at a sufficiently short interval such that thereshould be no appreciable readout value change due to actual radiationabsorption. Therefore, any readout value change between very shortintervals is the result of sensor or readout noise. Such noise power canthen be identified and filtered.

DESCRIPTION OF DRAWINGS

FIG. 1 shows a prior art of a system having a single ADC shared by thewhole pixel array. Since ADC is shared, all the pixel ADC operations areperformed sequentially.

FIG. 2 is also a prior art showing the column based ADC. An ADC isshared by all the pixels in the same column. All the column ADCoperations are done in parallel. But it's still sequential operationwithin a column.

FIG. 3 shows the in-pixel ADC. In this case each pixel contains an ADCand a sense amplifier (SA) is shared by pixels in the same column

FIG. 4 is a block diagram showing the microbolometer array, in-pixel ADCincluding a common Digital to Analog Converter (DAC), and otherinterface blocks.

FIG. 5 shows the current flow through a microbolometer pixel over time.The charge stored in Cstore in FIG. 7 is an integration ofmicrobolometer current over time.

FIG. 6 shows the DAC output (reference input to in-pixel comparator)schedule versus integration time.

FIG. 7 is a circuit diagram of the pixel based on constant voltage. Thisvoltage is converted to a variable current through a microbolomer, whichis stored in Cstore during integration. The Cstore is connected to asource follower and followed by a differential amplifier and anotherstage of amplication. The output is connected to the bit line throughthe row_en switch.

FIG. 8 shows the system with a constant current source. The voltageacross the microbolometer is captured and stored in Cstore. The sourcefollower is not implemented; instead a capacitor to store the comparatoroutput is shown.

DETAILED DESCRIPTION

This description relates to design, architecture, and implementation ofimaging devices based on Micro-Electro-Mechanical Systems (MEMS)microbolometer structures integrated with CMOS circuits. The inventionspecifically relates to design, architecture, and implementation ofreadout integrated circuitry that has significantly improved performanceand yield of the imager.

Microbolometer detectors sense incident electromagnetic radiation by thetemperature increase caused by the radiation's absorption in a sensingelement. The sensing element includes a temperature-sensing materialwhose resistivity is dependent on temperature. The temperature (orrather temperature change) of the element then can be read-out bymeasuring the resistance change of sensing element using associatedcircuitry. Either a constant current or constant voltage source convertsthis resistance change to a readout voltage or current respectively.This resultant voltage or current is integrated in a capacitor for acertain time. The resulting capacitor charge or voltage is then sampledand converted to a digital signal through an ADC.

Infrared imager is composed of a large number of the sensing elements orpixels that are arranged typically in two dimensional array asschematically shown in FIG. 1. The sensing elements 110 are connected tothe row decorder 120 by the set of electrical lines 150 and to columnmultiplexer 130 by another set of electrical lines 160.

Chip-level ADC architecture, in which one or a few ADCs 140 are used toreadout the whole array, is the prior art approach for arrayed imagesensor readout. In this method, pixels are readout one-by-one, serially,limiting the frame rate and pixel number (array size).

In visible-light image sensors, column-parallel ADC is oftenimplemented, as shown schematically in FIG. 2. In this case, the sensingelements 210 are again interconnected with the set of electrical lines250 to the row decoder 220, in the similar way as in chip-level ADCarchitecture. These sensing elements 210 are also interconnected withanother set of electrical lines 260 to analog storage circuitry 270which is in turn connected to the column ADCs 240. This improved priorart architecture allows simultaneous readout of signals from all columnsand increases readout speed over chip-level ADC. The samecolumn-parallel processing method can be applied to microbolometer-basedimage sensors. While this accommodates increasing array size and higherframe rate to some extent, the frame rate is still limited to the singleADC time multiplied by the number of rows. This practically limits themaximum frame capture rate to the imager refresh rate which is typicallyless than 30 frames per second, leaving no room to perform oversamplingthat is needed to process extra noise filtering and dynamic rangeextension. In addition, the readout time is a function of the arraysize, so large arrays also require faster ADC that consume higher powerand result in higher noise.

Further performance improvements can be achieved by including ADCs witheach sensing element, as illustrated in FIG. 3. For microbolometer-basedinfrared imaging, pixel size is governed by the microbolometer structuresize. Microbolometer structure reduction is limited by diffraction whenthe pixel dimensions approach the wavelength of detected light and bysensitivity losses because of increased noise in small pixels.Currently, state-of-the-art pixel pitch is 17 um and is generallyexpected to shrink to the diffraction limit of 10-12 um in the future.The relatively large area of imaging pixels in comparison tovisible-light pixels gives flexibility to readout circuit implementationin the area underneath the individual sensing elements.

In addition, in visible-light sensors, the photodiode sensor area isgenerally shared with readout circuitry limiting readout circuitry to afraction of total pixel area. For microbolometers, the pixel areaallowed for circuit implementation is not constrained by or competingwith the sensor area since the readout electronics are generally placedunderneath the sensor element.

Furthermore, in visible-light imagers, the pixel area limitation is evenworse since the routing of interconnect metal or polysilicon lines isconstrained such that it must not obscure the photodiode from incominglight. Whereas, in microbolometer technology, the entire pixel area canbe utilized for readout circuits, because sensors reside above readoutcircuits with vertical stacking.

Because of relatively large pixel size and the physical separation ofthe readout circuitry from the infrared sensor element and interconnect,the silicon area under the sensor element is generally not fullyutilized. In one embodiment of the invention, we describe methods to usethis area by placing parts of the ADC circuitry under the sensing pixelssuch that each pixels has its own ADC and therefore digitization ofsignals from all pixels can be performed simultaneously.

The schematic diagram of the pixel level ADC architecture is illustratedin FIG. 3. The sensing elements 310 are again interconnected to the rowdecoder 320 with the set of interconnecting electrical lines 350. Eachsensing elements is also connected to its own ADC 340. The readout ofthese ADCs is enabled through the sensing circuits SA, 380. It should beunderstood that the sensing elements are occupying the whole area andthat the interconnecting electrical lines 350 and 360 and ADCs 340 areplaced underneath of the array of the sensing elements 310, even thoughthey are shown at one level on the drawing in FIG. 3.

For the pixel based ADC, an illustrative example of the simplified pixelreadout schematics for global shutter operation is shown in FIG. 4. Atthe beginning of frame capture, all integration capacitors are reset andthen begin storing charge from the current driven through themicrobolometer sensor elements. After a short sub-sample time, thevoltage of the integration capacitor is sampled, and the result isstored in a storage capacitor. The voltage of the storage capacitor isconverted to a binary number through the in-pixel ADC. The digitaloutput of the ADC operation, a binary number, is the result of thecomparator operation. Each comparator output in a column is connected tothe bit line one row at a time and the bit line data is fed into thecolumn based sense amplifier 450. If the output changes from 0 to 1,this indicates that the sample capacitor voltage level is matched withthe reference DAC value. In this case, the DAC digital input values arestored in the corresponding memory location in the frame buffer 490 andthe status flag is set to 1.

The main consideration in this approach is that the in-pixel ADCconversion time should be sufficiently fast for ADC circuitryimplemented within a single pixel area. The conversion time can be asfast as tens of microseconds at an ADC precision of 10 bits. If moreprecision is needed, the conversion time doubles per additional bitprecision. This approach can be implemented in visible-light sensorsgiven enough silicon area; however, it is usually not practical sincethe pixel area is typically very limited. However, in microbolometerarrays, the pixel area is significantly larger and limited by sensingelement size; therefore, microbolometer pixels are ideal candidates forpixel-level ADC implementation.

In this pixel-level ADC embodiment, the analog output voltage of thesensor is compared with the input DAC value which is stored in on-chipnon-volatile memory. The output of a comparator, either high or low, isfed into a sense amplifier 490 located outside of the array in apreferred embodiment. This digital output is not necessarilyrail-to-rail but should be able to toggle a CMOS switch with enoughnoise margin between high and low levels. This output signal is thenconverted to a standard logic signal through a voltage or current basedsense amplifier 450 and then stored. In one embodiment, there is asingle sense amplifier per column to read the digital output from a bitline. In this case, the total readout time is simply the single ADCconversion time plus the column readout time multiplied by the number ofrows.

In this embodiment, the speed of the ADC operation is limited by thenumber of comparison steps, the DAC signal settling time, the speed ofthe comparator, and the sense amplifier settling delay. Since each pixelcontains a comparator, the comparator delay is lower than that of ashared comparator where multiple source followers are connected inparallel giving larger loading.

The sense amplifier settling delay can be minimized especially if it iscurrent switched. Since the sense amplifier operation in the columnreadout can be done at the rate of tens or hundreds of millions samplesper second, readout rate is not limited by column readout.

If the pixel DAC input to the comparator is required to settle to 95% ofthe target value, the settling time may be as high as hundreds ofnanoseconds since the DAC has to drive many nanoFarads of loading. Thus,if limited by DAC settling delay, the total ADC operation might take asmuch as 0.1 msec (1000 thermometer code steps×100 nsec) per frame. Thisapparently limits the readout frame rate to ten thousand frames persecond.

To improve conversion speed, either the DAC settling time needs to beshortened or the number of conversion steps should be reduced. Thesettling time can be minimized by optimizing the DAC driver, the wireloading, and the input capacitor of the comparators. To reduce thenumber of ADC conversions, the binary search method can be applied. Thismethod reduces the number of conversion cycles to a half compared to athermometer-coded conversion. For example, a single slope or thermometercoded 10 bit ADC will require 1024 conversion cycles, while the binarysearch method will require 512 conversion cycles. The trade-off ishigher noise due to large step change.

Unlike the chip based and column or row based parallel ADCs, the pixelbased ADC transfers the digital signal through the bit line. This willreduce the error caused by readout noise introduced by charge sharingand the settling time that occurs when analog signal is transferredthrough the bit line in shared ADC architectures.

Finally, since the readout is done for all pixels simultaneously, thereadout time is independent of array size thereby enabling large arrayswithout impact on frame rate.

Although the nominal sensor resistance value is fixed once the chip isfabricated, the amount of charge delivered to the integration capacitorcan be programmed to optimize sensor performance for each pixel whenapplying in-pixel ADC architecture. Specifically, the value of constantcurrent or voltage that converts the bolometer resistance change intovoltage or current pulses can be adjusted on a pixel-by-pixel basisaffording unique capabilities and advantages. These constant biascurrents or voltages, can affect the sensor performance in the areasof 1) time constant, 2) electrical dynamic range, 3) noise performance,and 4) power consumption.

In the preferred embodiment, the charge stored in the integrationcapacitor is not shared nor perturbed by the sampling operation.Instead, it continues to accumulate until the frame reset operation. Theintegration capacitor can be continuously sampled for further processingin this multi-sampling approach. The charge sampled in the storage capdoes not change during the ADC and readout operations, and therefore canbe frequently sub-sampled during an integration cycle allowing complexfunctionality such as dynamic range expansion and noise reduction viaauto-correlation.

In traditional readout using linear dynamic range, a single image framecontains signals ranging from the highest to the lowest value of thedynamic range occupying the full range of ADC. In one embodiment,in-pixel ADC architecture is used to expand the dynamic range using themulti-sampling capability described above. In this example, as depictedin FIG. 4, all integrating capacitors are reset, then start integratingat the same time, but each pixel has a unique integration time based onthe pixel's responsiveness and its brightness in the image as determinedby sub-sampling the pixel. This is done by the pre-programmed DAC outputlevel that is compared with the storage cap values; the resultantcomparator output is read out. If the output changes the polarity for aparticular pixel—the flag value changes from 0 to 1—then the DAC inputvalues are stored in the corresponding frame buffer location along withthe status bit. For this pixel, the frame data is extracted. However,for the rest of the array for which the status bits are not set, the ADCoperation continues until all the pixels are compared with the entireDAC range.

In a standard approach, the ADC operation is done as fast as possiblefrom one bit to the next at a fixed time interval giving a linear,uniform dynamic range. However, in this embodiment, the dynamic rangecan be expanded for a certain region of interest by extendingintegration time intervals for pixels of certain intensity. For example,if the human body is of interest, the ADC range and therefore depth ofgrey scale in the vicinity of 37 C (human body temperature) is expanded.This will improve the picture quality and recognizability of a humansubject, while deemphasizing the background. As in FIG. 5, ADC samplesfrom t1 to t2 (from DN=2n to DN max, where DN represents DAC inputvalue) as fast as possible deemphasizing this range, but from t2 thrut3, sampling is done at much larger incremental time steps. The chargestored in the integration capacitor that is sampled at different timesis proportional to the integration of the current that indicates thecurrent change, which is in turn inversely proportional to theresistance change of microbolometer. The average rate of change of themicrobolometer is simply the integration of current throughmicrobolometer over the integration time which is equivalent to thedigitized integration capacitance value, divided by the integrationtimes.

Theoretically, time intervals can be set arbitrarily, so that thedynamic expansion level of the bolometer is done at any precision. If anoutput with a linear scale is desired, the number of bits can beincreased beyond the DAC range in the frame buffer, so that the binaryzeros are tagged to the pixels in the un-extended range. Although ADC'scomparison steps from t1 to t2 and t3 to t4 are preceded as fast aspossible, the delays between the bit comparisons can be compensated inthe slope calculation. Once all the bits of a full DAC range arecompared and all frame buffer locations are filled, the current framecapture is completed, the next frame capture will start, and the sameprocedure will repeat again. Extension of the dynamic range is shown inFIG. 5, where the average rate of resistance change with dynamic rangeextension is plotted and compared with a linear dynamic range.

Microbolometer sensitivity is limited by the sensor's noiseperformance—specifically 1/f noise originating from the sensor materialand small volume of that material used as the thermistor, resistor thatchanges its resistance with temperature. The problem is particularlysevere in microbolometer technology and mitigating this problem has beenone of the main challenges of the microbolometer IR imagers. New sensordesigns and materials have been the focus to reduce 1/f noise, but novelelectronic readout approaches can be also very beneficial.

The relationship between the reference voltages versus the detector'sultimate sensitivity—the noise equivalent temperature difference(NETD)—is such that the higher the sensor bias voltage, the lower thetemporal NETD. However, the higher the voltage across the microbolometeris applied, the higher the current flow through the microbolometer willbe. This translates to undesirable self-heating effects and higheroverall power consumption. Hence, the optimal bias voltage setting willbe a trade-off between the low noise and low power consumption. As inthe pixel circuit diagram, FIG. 7, the bias voltage across themicrobolometer sets the integration current lint. This can beapproximated as follows:

lint=(Vin_bias−Vt_sf)/Rmb,

Where Vin_bias is the input bias voltage,Vt_sf is the threshold voltage of the source follower,Rmb is the resistance of the microbolometer

The same principle can be applied to the pixel system depicted in FIG.8. In this implementation, a constant current, instead of constantvoltage, is applied to the microbolometer which is equivalent to avariable resistor. As mentioned above, the higher the current flowsthrough the microbolometer, the higher the voltage across themicrobolometer, as well as the dynamic range of the voltage output andthe signal-to-noise ratio will be.

Noise in a microbolometer IR sensor is mainly comprised of fixed patternnoise, 1/f noise and thermal noise. In the following embodiments, threenoise reduction solutions are proposed—a digital Correlated DoubleSampling (CDS), signal averaging, and 1/f noise reduction based onautocorrelation and adaptive filtering. Each of these methodscomplements the shortcomings of the other methods. In particular, forsensors with in-pixel ADC, these methods are much more effective due tomultisampling and high frame capture rate.

Analog and digital signal processing both during and after the imagesare captured and stored can be applied to reduce image noise. For signalprocessing and noise filtering, CDS or auto-correlation techniques canbe used. CDS, which itself is a simple autocorrelation technique, cancorrect 1/f noise only to a limited level. Other auto-correlationtechniques are especially promising if a high level of noisecancellation is required.

High pixel sampling rates possible with in-pixel ADC uniquely enableseffective auto-correlation for 1/f noise reduction. In order to fullyutilize the autocorrelation technique for the 1/f noise cancellation,the sampling rate needs be high—the sampling bandwidth should be widerthan the 1/f noise frequency spectrum, so that the result will cover theentire 1/f noise bandwidth.

CDS (Correlated Double Sampling) is often used for noise cancellation inan IR image capture system. In this proposal, the following procedure isused. 1) With the shutter closed, after settling time, a voltage acrossthe microbolometer element 710 in FIG. 7 is sampled and stored in thecapture capacitor 720. The capture capacitor voltage is read out throughan in-pixel ADC and the digital outputs are stored in a memory array.The row by row readout continues until the last (bottom) row iscomplete. This completes the capture of the reference frame. 2) Thecapture capacitor 720 is reset again and all in-pixel current sourcesincluding the microbolometer current mirror and the bias current mirrorsfor the amplifiers are powered down. This will reduce the powerconsumption and prevent the devices from accumulating noise. 3) After apreset integration time, the current sources are powered on, capture isrepeated and storing and ADC conversion procedure is completed. Thepreviously stored value is then subtracted from the new digital value.This sequence is repeated for next read until finishing the entireframe. In a multisampling system, the second and third procedures arerepeated, and the dark value stored in the memory is subtracted againfrom the newly captured values. The difference accounts for the netcurrent change due to the temperature change in the bolometer as aresult of exposure to the IR light. All other background signalsincluding the differences in resistance from one bolometer element toanother, capacitor mismatch, thermal noise and variations in amplifiergain are cancelled out. Although this scheme will mainly subtract thefixed pattern noise, it also reduces some white noise.

Advantages of the digital CDS proposed in this embodiment include beingfree from errors caused by mismatching of two or more capacitors andother noises incurred during readout including the current leakage fromthe capacitors and extra charge coupling during conversion.

Although the CDS scheme cancels the fixed pattern noise and reduces thewhite noise including 1/f noise like DC offsets and low frequency noise,its effectiveness with regards to 1/f noise is limited and dependent onthe bandwidth of the sampling frequency. If the sampling frequency islower than the corner frequency of 1/f noise density function, the highfrequency noise will not be corrected.

In order to supplement the shortcomings of CDS and reduce any residualnoise after CDS, extra post-process digital filtering is proposed. Asimple and popular filtering scheme is signal averaging, which smoothesout all high frequency noise. However, averaging causes blurring ofsharp edges. The next description will help reduce 1/f noise as well asthermal noise, and improve the edge profile.

Since the temporal noise, including the 1/f noise, is time-variant, theframe by frame data are stored and processed, where the pixel-to-pixelvalues in fixed time interval are compared, averaged and filtered.Averaging, which has normally bandpass nature, will reduce the highfrequency noise and is effective in reducing noise such as thermal noisebut is not as effective in reducing 1/f noise.

Reduction of residual 1/f noise after CDS is achieved by a post digitalnoise filtering based on adaptive filtering and autocorrelationfunction. A method is described as follows. First, the noise signalspectral density is calculated based on autocorrelation method.

Discrete Fourier Transform of the power spectrum density is theautocorrelation function. If temporally separated by sampling period,the pixel values are stationary, and the autocorrelation function valuesshould be 1. If the sampling frequency is fast enough, captured sceneswon't change much. The autocorrelation function will still be closeto 1. If any white noise is added to the signals, the autocorrelationfunction will be reduced, where the autocorrelation function of the purewhite noise with a flat frequency response is 0. 1/f noise is bandlimited, but the autocorrelation function follows similarly to the whitenoise. 1/f noise that has a long time constant behaves like a fixedpattern noise and can be cancelled by CDS in time domain. The highfrequency portion of the white noise is reduced by averaging. In amethod of this embodiment, in order to reduce the residual noise andincrease the overall frame Signal/Noise ratio (S/N) is to keep signalmagnitude of the pixel with high correlation factor, but attenuate thepixel signals with low correlation factors. If the signals areamplified, less gain is assigned to the latter.

The post signal processing utilizing filters like Wiener filter canfurther reduce the noise. The signal with embedded noise is alsoconverted to a frequency domain by a Fourier transform. Also, theautocorrelation of the noise estimate is calculated by the DigitalFourier transform. These values are converted back to time domain by theInverse Digital Fourier Transform and used to calculate the filtercoefficient. An expected or estimated output is compared with the outputof a filter and the error is observed. This error is then minimized bythe adjusting the filter coefficient. One of the algorithm used to findthe next coefficients can be LMS (Least Mean Square), that is,coefficients are changed to minimize the mean square error between thefiltered output and the input signals coupled with noise.

One advantage of a digital system with an in-pixel ADC, as proposed inthis embodiment, is that the calibration data captured and stored in anonvolatile memory during final testing are compatible with any readoutstage output. Hence, the data can be used as feedback signals tocompensate component mismatches, decrease noise and extend the dynamicrange.

Initial calibration needs to be performed during the test stage and canbe done as follows:

First, at room temperature, with the shutter closed, the settled voltageacross the bolometer with the constant current source (sample andmeasure voltages of the storage capacitor) is measured.

Next, all voltage levels are recorded and the measurement with a knownIR intensity is taken. Per pixel difference is the response of abolometer and that allows calculation of thermal coefficient ofresistance (TCR) from the results. In addition, the mean value iscalculated and subtracted from the pixel value. These differencesrepresent the variation of bolometer sensitivity and are recorded into anonvolatile memory. These numbers are used for compensation during realimage capture. Individual pixel TCR variations are recorded. The sameprocedure is repeated at an elevated temperature, for example at 70 degC. The results are also recorded in the nonvolatile memory.

During normal usage, the chip temperature is measured through an on-chipbandgap or a junction diode temperature sensor. The stored sensitivityvalues in the nonvolatile memory are interpolated, or extrapolated ifthe temperature is out of the reference range used for calibration. Whenthe sensitivity of a bolometer or TCR is higher, the bolometerresistance change will be higher, but the voltage change across thestorage cap will be lower with the constant current flow through thebolometer. Thus, in order to equalize the response, the storedsensitivity variations are simply multiplied by the sampled data. Thiscompensation can be performed before the captured data is stored in thememory and doesn't require any extra memory space. This calibrationmethod will be also used to screen out the dead pixels, opens or shorts,or any part of readout circuits and path, and hot or cold pixels. Thehot pixels are pixels that are too sensitive and the cold pixels haveinsufficient sensitivity. The dead pixels are recorded in thenonvolatile memory and excluded during normal usage. These pixel valuesare estimated based on neighboring pixel values. The values areinterpolated or averaged between neighboring pixels or the edge rule ifthey are located at the transition area. If the rendered image of acluster of dead pixels is visible and a multiple number of clusters aredetected, these defective arrays can be screened out. Definition of acluster and the threshold of the number of clusters to fail arepredetermined and also stored in the nonvolatile memory. Since theproposed per pixel ADC can compensate per pixel difference, and hot orcold pixels can be further compensated using an extended bias current,or voltage range. Additionally, ADC range can be extended if the circuitcan tolerate it. Otherwise, the ADC range can be extended in expense ofprecision. In this way some hot and cold pixels can be salvaged, andthus the yield can be raised. Since input voltage (or current) levelsaffect the dynamic operating range, noise and power levels, the average,max and mean output levels are measured or calculated over temperaturerange, and the optimal input levels are derived from these values. Thesevalues are stored in nonvolatile memory and used in image capture.

The auto-correlation signal processing and filtering is done in thecompanion image processor which is advantageous since modern digitalsignal processing can be done fairly cheaply and with high performance(density, power and speed). Furthermore, since all the signalstransferred and processed beyond the pixel are in the digital domainwhen using in-pixel ADC, they are immune to additional noise coupling(assuming the signal transfer out of the imager chip to the signalprocessor or equivalent are error free).

The multiple frame data received by the image processor are averagedpixel by pixel from frame to frame to remove the temporal noise. Imagesare smoothed by spatial low pass filtering along with the edgesharpening process where the high frequency component is maintained.

For the multiple reference black columns, the IR is blocked so that theoutput simply represents the microbolometer resistance that varies withthe background temperature and heat conduction.

In a system with the pixel-based ADC, comparator operations for allpixels are done at the same time, thereby providing global shuttercapability. The pixel-based comparator changes state when the inputvoltage passes the DAC voltage threshold. At that instant, thecorresponding digital input data to DAC is buffered, then stored in theframe buffer, and the status flag is set to indicate the completion ofconversion of that pixel. The ADC operation is continued until the fullrange of DAC output is compared with the stored voltage and all pixelstatus bits are set for completion.

Since a global shutter captures and stores an instantaneous image, it isuseful in capturing scenes containing motion without image artifactsthat are characteristic of sensor arrays using a scrolling shutter. Ifrows or pixels in an array are sampled at different times (as incolumn-parallel ADC), a fast moving objects can exhibit phenomenon suchas wobble, skew, smear, partial exposure, and aliasing that degradeimage quality and can cause image processing algorithms to fail.Furthermore, the dynamic range extension methods discussed earlier canonly be implemented when using global shutter since it requiresdetermining the full brightness range of an image at one instance intime. Global shutter can be implemented using two different methods: thefirst uses global sampling and storage of analog voltages along withshared ADCs in chip level or column level based system, the second usespixel-based operations where the ADC operation is done globally.

In a shared ADC system, the ADC operation is performed pixel by pixel orrow by row sequentially (i.e. one pixel or row at a time). In order toimplement a global shutter it is necessary to sample and store all thepixels at the same time. However, since there is a time lag between oneADC operation to the next, variation of the individual capacitancevalue, large leakage and coupling of thermal noise of the switchingtransistor are problematic. Because the thermal noise and capacitancevariation are usually in a fixed pattern, the noise can be cancelled bythe CDS operation previously described. However, the leakage cannot befiltered with CDS. Moreover, if the time lag between the first and lastADC operations is large, the leakage will considerably affect overallsystem performance.

With in-pixel ADC, the whole array only requires one ADC time, althougha readout time is required per row. Since readout time is donedigitally, precise settling of bit line voltage is not necessary and thetime to complete whole frame readout, compared to the ADC conversiontime per pixel or row, can be orders of magnitude shorter than in thesystem with shared ADCs.

In an alternative embodiment, it is possible to include a storagecapacitor to store the comparator output where the stored value is adigital binary number instead of an analog voltage that is aninstantaneous sample of the integration capacitor. Since the storedvalue is digital, the leakage of storage capacitor which is placed afterthe in-pixel comparator does not affect the outcome as much as an analogoutput system. Furthermore, the sampled analog value of the integrationcapacitor does not need to be stored and the system S/N will not bedependent upon the artifact created by sampling and storing of thestorage capacitor signal value. If the sampling frequency is fast,errors caused by the delay due do a global ADC and frame readout will bevery small. However, in order to minimize this delay, each bitcomparison time as well as the number of comparison steps need to beminimized without sacrificing the precision and dynamic range.

The option implementing a current based readout is depicted in FIG. 8.In this figure, a constant current flows through the microbolometer, thevoltage across the bolometer pixel 810 is sampled and stored in Cstore820. The ADC operation starts and the result of each binary weighted bitcomparison is stored in Cread_hold capacitor 830. After reading thewhole frame, ADC moves to the next most significant binary bitcomparison until all bits, DAC output, are compared and the outputsstored. The DAC outputs can also be thermometer coded in order to reducethe switching noise. In this case, the frame time will increase becausethe number of comparisons will increase. In the case of FIG. 8, whilethe bit lines are being read out, the current sources to the amplifierscan be switched off in order to save power.

In a multi-sample system, the storage capacitor is sampled again and thewhole ADC operation is repeated. Otherwise, the storage capacitor isupdated at the beginning of next frame readout.

In FIG. 7, a constant voltage mode is depicted, and the figure shows thepossibility of sharing a part of differential amplifier as well as thefollowing stage amplifiers by a multiple pixels in order to save thecircuit area and power.

In a column-based ADC system, the sequence is the same, except the ADCsare placed outside of the array and the ADC operation is done row byrow. The storage capacitor output is connected to the bit line through asource follower. This bit line voltage is sampled and fed into thecolumn-based ADC. At the ADC input, the bit line voltage can be sampledin another sampling capacitor which is shared by all the pixels in thesame column. However, if each pixel has a storage cap, an extracapacitor in the ADC is not always necessary. This is because the bitline voltage will be held steady during a full ADC conversion period.

When multiple samples per frame are not needed, the storage capacitorcan be omitted for a global frame capture. The integration capacitor 740is sampled in lieu of the storage capacitor 720, and the output is fedinto the bit line through a source follower directly. ADC conversion forthis analog input is performed for all bit levels, and after completionof one row ADC, the next row is selected and the ADC conversionprocedure is repeated until the last row. As the in-pixel ADC, thecolumn based ADC reference inputs and analog signals, can be generatedeither in binary coded sequence, for example from most significant bitto least significant bit as in a successive approximation ADC, or inramp fashion, as in a single slope ADC. Any other type of ADC, includinga sigma-delta, successive approximation, or cyclic converter is alsopossible for column based implementation.

As the IR imager cost approaches that of a visible image sensor, withimproved resolution and image quality, as well as high frame ratecapture and global shutter capability, new possible applications emerge.One such application is gesture recognition. Existing technologieseither rely on time-of-flight or the emission of near-IR laser dots torecognize 3D structure. These technologies require a light source thatis invisible to the human eye at wavelength in the 850-950 nm range.Although these technologies perform well, the light source consumessignificant power. High illumination intensity is often needed toprovide adequate signal-to-noise ratio if the ambient brightness is highand, the system often requires a cooling fan. Also high light intensityin the visual band can bloom or smear into the Near IR band,deteriorating the image quality and necessitating visible lightfiltering components.

In contrast, a microbolometer IR sensor is passive, relying on thedetection of distinctive infrared radiation emitted by warm objects;therefore, it does not require an external light source. In a typicalscene, the human body temperature is usually easily distinguishable fromthe background, because the body temperature of a person is relativelyconstant and uniform around 37 deg C. and the emissivity of the bareskin is approximately 0.98, and consequently there is little variationin filtering the images of human body or parts out of a captured image.This is true for both indoor and outdoor imagers. Especially the exposedbody parts like hands or a face can be readily isolated and identifiedfrom the background and the detailed motion can be analyzed. An easierand preferred way of analyzing the motion is to use the symbolicpresentation instead of three dimensional (3D) image reconstructed fromthe time-of-flight methods. As image quality improves and more detailedimages become available, temperature contour maps and data points suchas finger tips or valleys in between fingers can be extracted, as donein facial or finger print recognition systems, and used for 3D analysis.By these symbolic lines and points along with a fast frame capture, the3D motion tracking can be done without costing too much memory space,power and computation time.

IR recognition systems do not require an extra light source and aresuitable for low power portable devices.

The imaging array and processing electronics described above arecombined with the optical elements and control electronics to form theoptical imaging systems such as cameras for imaging applications in theinfrared spectral range.

Those skilled in the art will recognize that such design, architecture,and implementation of readout electronics can be applied to a multitudeof devices that require readout of an array of sensors.

What is claimed is:
 1. A suspended sensing element encapsulated withoutmechanical contact to the element, having readout integrated circuitlocated underneath the element.
 2. The device of claim 1 with integratedcircuit containing analog to digital converter.
 3. Array of elements ofthe devices of claim
 1. 4. Array of elements of the devices of claim 2.5. Readout integrated circuit of claim 1 with programmable sensorprobing bias voltage or current.
 6. Readout integrated circuit of claim1 with continuous sampling of integration capacitor for dynamic rangeexpansion and noise reduction.
 7. Readout integrated circuit of claim 1with subsampling of device elements each with unique integration time.8. Readout integrated circuit of claim 1 with correlated double samplingto cancel noise.
 9. Readout integrated circuit of claim 1 with autocorrelation for digital noise filtering.
 10. Readout integrated circuitof claim 1 with global shutter for very fast image formation.
 11. Signalcorrecting and processing unit for suspended sensing elements,encapsulated without mechanical contact to the elements and havingreadout integrated circuit located underneath each element, the unithaving memory containing sensitivity of all elements, their thermalsensitivity and algorithms to interpolate between nonfunctionalelements.
 12. Signal processing unit of claim 11 that is on the samechip as the sensing elements.
 13. Signal processing unit of claim 11that is on one chip while the sensing elements are on the separate chip.14. Electronic circuitry for image analysis and recognition based onprocessing signals from the signal correcting and processing unit ofclaim
 11. 15. An infrared camera consisting of array of elements ofclaim 1, optical elements, control electronics and digital imageprocessing chip.
 16. The system based on elements of claim 1 combinedwith the motion algorithms to enable gesture recognition.